Developing High-Level Irregularly-Parallel Programs for Multiple Architectures

The variety of parallel architectures nowadays available implies that models for parallel software development should deliver acceptable performance on a range of architectures with minimal development effort. One approach is to use languages with very high level, and substantially architecture-independent, specification of parallel coordination. This paper presents a novel profiling-based methodology for developing irregularlyparallel applications for multiple architectures in languages with high-level coordination. The methodology has two main phases: an architecture-independent phase of idealised parallelisation, and an architecture-dependent phase of accurate performance prediction. The methodology has been distilled from the development of a dozen applications on various parallel machines. We have previously achieved good speedups with only minimal code changes: a speedup of 12.0 on a 16-processor workstation cluster and 2.8 on a 4-processor Sun SMP for a data-intensive program, and 11.9 on a 16-processor cluster for a symbolic computation application. The methodology is illustrated by developing a substantial application for two very different architectures: a Beowulf cluster and a Sun SMP. Seven alternative parallel versions of the program are developed and evaluated using a simulated idealised architecture. Realistic simulation of the two target architectures accurately predicts the version that delivers the best performance. While the speedups achieved for this program are modest, 7.5 on a 30-processor Beowulf and 1.8 on 1School of Mathematics and Computer Science, Heriot-Watt University, Edinburgh, UK; Phone: +44 (0)131-451-3435; Fax: +44 (0)131-451-3732; Email: {ceemka,hwloidl,trinder}@macs.hw.ac.uk